Meta-analysis of the effect of consistency on success in early learning of programming

A test was designed that apparently examined a student’s knowledge of assignment and sequence before a first course in programming but in fact was designed to capture their reasoning strategies. An experiment found two distinct populations of students: one could build and consistently apply a mental model of program execution; the other appeared either unable to build a model or to apply one consistently. The first group performed very much better in their end-ofcourse examination than the second in terms of success or failure. The test does not very accurately predict levels of performance, but by combining the result of six replications of the experiment, five in UK and one in Australia. we show that consistency does have a strong effect on success in early learning of programming. Background programming experience, on the other hand, has little or no effect.